Method and system for assessing viewing quality of media objects

ABSTRACT

Method, system, and programs for assessing viewing quality of media objects (e.g., advertisements) are provided. In one example, information regarding one or more delivery conditions of an advertisement that is served at a user device is received. Viewing quality of the advertisement is estimated based on the one or more delivery conditions. Information regarding one or more advertising pricing parameters related to the advertisement is received. The viewing quality of the advertisement and the one or more advertising pricing parameters are provided.

CROSS REFERENCE TO RELATED APPLICATION

This application is a division of U.S. application Ser. No. 14/586,015filed Dec. 30, 2014, which is incorporated herein by reference in itsentirety.

BACKGROUND

1. Technical Field

The present teaching relates to methods, systems, and programming foradvertising. More particularly, the present teaching relates to methods,systems, and programming for assessing advertisement delivery quality.

2. Discussion of Technical Background

In recent years, the percentage of the consumer demographics on theInternet has experienced exponential growth, and continues to grow,thereby fueling the incentive for businesses to advertise online. Manyadvertising companies offer pay-per-impressions programs, which chargetheir customers (e.g., businesses) for each rendering of anadvertisement. Per-per-impression programs, however, fail to considerwhether an advertisement is actually viewable to a user when anadvertisement has been considered delivered, such as whether theadvertisement is in-view on a screen to the user. Similar problems existfor general media objects delivered to users for viewing, includingvideos, animations, images, audios, textual content, other types ofmedia objects, or any combination thereof. While existing systems thatassess viewing quality of advertisements alleviate some of the issues ofpay-per-impression programs, such systems are nevertheless amendable toimprovement. As an example, many existing systems that assess viewingquality of media objects will fail to mark an advertisement on a browseras viewable even when the advertisement is in-view if the browser is notthe active application currently used by a user. These and otherdrawbacks exist.

SUMMARY

The teachings disclosed herein relate to methods, systems, andprogramming for assessing viewing quality of media objects (e.g.,advertisements).

In one example, a method, implemented on at least one machine, each ofwhich has at least one processor, storage, and a communication platformconnected to a network for assessing viewing quality of media objects,is disclosed. Information regarding one or more delivery conditions ofan advertisement that is served at a user device is received. Viewingquality of the advertisement is estimated based on the deliveryconditions. Information regarding one or more advertising pricingparameters related to the advertisement is received. The viewing qualityof the advertisement and the advertising pricing parameters areprovided.

In another example, a method, implemented on at least one machine, eachof which has at least one processor, storage, and a communicationplatform connected to a network for assessing viewing quality of mediaobjects, is disclosed. Information regarding one or more frame rates ofan advertisement that is served at a user device is received. Viewingquality of the advertisement is estimated based on the frame rates. Theviewing quality of the advertisement is provided.

In another example, a method, implemented on at least one machine, eachof which has at least one processor, storage, and a communicationplatform connected to a network for assessing viewing quality of mediaobjects, is disclosed. Information regarding a state of each of one ormore objects associated with a media object that is served at a userdevice is received. Viewing quality of the associated objects isestimated, based on the state of each of the associated objects, withoutreceiving a frame rate of the associated objects. Viewing quality of themedia object is estimated based on the viewing quality of the associatedobjects.

In another example, a method, implemented on at least one machine, eachof which has at least one processor, storage, and a communicationplatform connected to a network for assessing viewing quality of mediaobjects, is disclosed. Information indicating a throttle event withrespect to each of one or more objects associated with a media objectthat is served at a user device is received. A parameter in connectionwith each of the associated objects is throttled by the respectivethrottle event. Viewing quality of the associated objects is estimatedbased on the throttle event with respect to each of the associatedobjects. Viewing quality of the media object is estimated based on theviewing quality of the associated objects.

In another example, a system for assessing viewing quality of mediaobjects is disclosed. The system comprises a viewing quality assessmentsystem. The viewing quality assessment system comprises a deliverycondition evaluation unit, a viewing quality evaluation unit, and abilling and/or reporting evaluation unit. The delivery conditionevaluation unit is configured to receive information regarding one ormore delivery conditions of an advertisement that is served at a userdevice. The viewing quality evaluation unit is configured to estimateviewing quality of the advertisement based on the delivery conditions.The billing and/or reporting evaluation unit is configured to receiveinformation regarding one or more advertising pricing parameters relatedto the advertisement, and provide the viewing quality of theadvertisement and the advertising pricing parameters.

In another example, the delivery condition evaluation unit is configuredto receive information regarding one or more frame rates of anadvertisement that is served at a user device. The viewing qualityevaluation unit is configured to estimate viewing quality of theadvertisement based on the one or more frame rates. The billing and/orreporting evaluation unit is configured to provide the viewing qualityof the advertisement.

In another example, the delivery condition evaluation unit is configuredto receive information regarding a state of each of one or more objectsassociated with a media object that is served at a user device. Theviewing quality evaluation unit is configured to estimate viewingquality of the associated objects based on the state of each of theassociated objects without the system receiving a frame rate of theassociated objects, and estimate viewing quality of the media objectbased on the viewing quality of the associated objects.

In another example, the delivery condition evaluation unit is configuredto receive information indicating a throttle event with respect to eachof one or more objects associated with a media object that is served ata user device. A parameter in connection with each of the associatedobjects is throttled by the respective throttle event. The viewingquality evaluation unit is configured to estimate viewing quality of theassociated objects based on the throttle event with respect to each ofthe associated objects, and estimate viewing quality of the media objectbased on the viewing quality of the associated objects.

Other concepts relate to software for assessing viewing quality of mediaobjects. A software product, in accord with this concept, includes atleast one machine-readable, non-transitory medium and informationcarried by the medium. The information carried by the medium may beexecutable program code data regarding parameters in association with arequest or operational parameters, such as information related to auser, a request, or a social group, etc.

In one example, a machine readable and non-transitory medium hasinformation recorded thereon for accessing viewing quality of mediaobjects, wherein the information, when read by the machine, causes themachine to perform a series of steps. Information regarding one or moredelivery conditions of an advertisement that is served at a user deviceis received. Viewing quality of the advertisement is estimated based onthe delivery conditions. Information regarding one or more advertisingpricing parameters related to the advertisement is received. The viewingquality of the advertisement and the advertising pricing parameters areprovided.

In another example, a machine-readable, non-transitory medium hasinformation recorded thereon for accessing viewing quality of mediaobjects, wherein the information, when read by the machine, causes themachine to perform a series of steps. Information regarding one or moreframe rates of an advertisement that is served at a user device isreceived. Viewing quality of the advertisement is estimated based on theone or more frame rates. The viewing quality of the advertisement isprovided.

In another example, a machine-readable, non-transitory medium hasinformation recorded thereon for accessing viewing quality of mediaobjects, wherein the information, when read by the machine, causes themachine to perform a series of steps. Information regarding a state ofeach of one or more objects associated with a media object that isserved at a user device is received. Viewing quality of the associatedobjects is estimated, based on the state of each of the associatedobjects, without receiving a frame rate of the associated objects.Viewing quality of the media object is estimated based on the viewingquality of the associated objects.

In another example, a machine-readable, non-transitory medium hasinformation recorded thereon for accessing viewing quality of mediaobjects, wherein the information, when read by the machine, causes themachine to perform a series of steps. Information indicating a throttleevent with respect to each of one or more objects associated with amedia object that is served at a user device is received. A parameter inconnection with each of the associated objects is throttled by therespective throttle event. Viewing quality of the associated objects isestimated based on the throttle event with respect to each of theassociated objects. Viewing quality of the media object is estimatedbased on the viewing quality of the associated objects.

BRIEF DESCRIPTION OF THE DRAWINGS

The methods, systems and/or programming described herein are furtherdescribed in terms of exemplary embodiments. These exemplary embodimentsare described in detail with reference to the drawings. Theseembodiments are non-limiting exemplary embodiments, in whichlike-reference numerals represent similar structures throughout theseveral views of the drawings, and wherein:

FIGS. 1-2 depict exemplary embodiments of a networked environment inwhich assessment of viewing quality of media objects is applied,according to different embodiments of the present teaching;

FIG. 3 is a high level exemplary system diagram of a viewing qualityassessment system, according to an embodiment of the present teaching;

FIG. 4 is a high level exemplary diagram of different deliveryconditions, according to an embodiment of the present teaching;

FIG. 5 is a high level exemplary system diagram of an advertisingdelivery system and a viewing quality assessment system, according to anembodiment of the present teaching;

FIG. 6 is a flowchart of an exemplary process of assessing billinginformation related to advertisements, according to an embodiment of thepresent teaching;

FIG. 7 is a high level exemplary system diagram of a delivery conditionevaluation unit, according to an embodiment of the present teaching;

FIGS. 8A and 8B illustrate exemplary presentations of advertisements,according to different embodiments of the present teaching;

FIG. 9 is a flowchart of an exemplary process of assessing viewingquality of an advertisement based on delivery conditions of portions ofthe advertisement, according to an embodiment of the present teaching;

FIG. 10 is a flowchart of an exemplary process of assessing viewingquality of an advertisement based on a frame rate of the advertisement,according to an embodiment of the present teaching;

FIG. 11 is a flowchart of an exemplary process of assessing viewingquality of a media object based on a state of an object associated withthe media object, according to an embodiment of the present teaching;

FIG. 12 is a high level exemplary system diagram of a viewing qualityevaluation unit, according to an embodiment of the present teaching;

FIG. 13 illustrates an exemplary presentation of an advertisement andits associated objects, according to different embodiments of thepresent teaching;

FIG. 14 is a flowchart of an exemplary process of assessing viewingquality of a media object based on a related throttle event, accordingto an embodiment of the present teaching;

FIG. 15 is a flowchart of an exemplary process of assessing viewingquality of advertisements, according to an embodiment of the presentteaching;

FIG. 16 depicts an architecture of a mobile device that can be used torealize the present teaching as a specialized system; and

FIG. 17 depicts an architecture of a computer that can be used torealize the present teaching as a specialized system.

DETAILED DESCRIPTION

In the following detailed description, numerous specific details are setforth by way of examples in order to provide a thorough understanding ofthe relevant teachings. However, it should be apparent to those skilledin the art that the present teachings may be practiced without suchdetails. In other instances, well known methods, procedures, components,and/or circuitry have been described at a relatively high level, withoutdetail, in order to avoid unnecessarily obscuring aspects of the presentteachings.

The present teaching relates to assessing viewing quality of mediaobjects (e.g., advertisements, videos, animations, images, audios, orother media objects). In various embodiments, viewing quality of a mediaobject (provided for presentation to a user) is estimated. In anembodiment, the viewing quality of the media object is provided forreporting to one or more business entities. As an example, theestimation is used to generate a report indicating the viewing qualityof the media object and/or determine billing information related todelivery of the media object. The estimation may, for example, includedetermining whether and/or the extent to which the advertisement isviewable to the user.

In an embodiment, information regarding one or more delivery conditionsof the media object is received. The delivery condition information isthen processed to determine the delivery conditions of the media object,including conditions of a network (e.g., bandwidth, latency, etc.) overwhich the media object is delivered, a response time of the mediaobject, a frame rate of the media object, a compression ratio of themedia object, or other delivery condition (e.g., whether a parameter ofthe media object has been throttled or other conditions). In someembodiments, the delivery conditions correspond to one or more giventimes at which portions of the media object is rendered (e.g., a portioncorresponding to a first time interval of the media object, a portioncorresponding to a second time interval of the media object, etc.).Viewing quality of the media object is estimated based on the deliveryconditions of the media object.

In an embodiment, the media object includes multiple portions where atleast some of the media object portions have delivery conditionsdifferent from one another. Receipt of information regarding thedelivery conditions of the media object includes receipt of informationregarding a delivery condition of a first portion of the media object,receipt of information regarding a delivery condition of a secondportion of the media object, and so on. In some embodiments, at leastsome of the media object portions have delivery conditions differentfrom one another at a given time. As an example, the delivery conditioninformation is processed to determine a frame rate of the first portionof the media object, a frame rate of the second portion of the mediaobject, and so on. Viewing quality of the media object is estimatedbased on the frame rate of the first portion of the media object, theframe rate of the second portion of the media object, or frame rates ofother portions of the media object. As another example, the deliverycondition information is processed to determine other deliveryconditions of the media object portions, and viewing quality of themedia object is estimated based on the other delivery conditions of themedia object portions.

In an embodiment, information regarding one or more objects associatedwith the media object is received. The associated objects may, forexample, include supplemental objects for monitoring aspects of themedia object. The associated object information is then processed todetermine one or more states of the associated objects, one or moredelivery conditions of the associated objects, or other informationregarding the associated objects. A state of an associated object may,for example, include a throttled-down state identifying that theassociated object (or a parameter thereof) is throttled down, athrottled-up state identifying that the associated object (or aparameter thereof) is throttled up, a non-throttled state identifyingthat the associated object (or a parameter thereof) is not throttled, orother state. A delivery condition of an associated object may includeconditions of a network (e.g., bandwidth, latency, etc.) over which theassociated object is delivered, a response time of the associatedobject, a frame rate of the associated object, a compression ratio ofthe associated object, or other delivery condition. Viewing quality ofthe media object is estimated based on the states of the associatedobjects, the delivery conditions of the associated objects, or othercriteria. In some embodiments, viewing quality of the media object isestimated based on the states of the associated objects withoutreceiving information identifying the delivery conditions (e.g., withoutknowing the frame rates of the associated objects, without knowingcertain other delivery conditions of the associated objects, etc.).

As discussed, although systems that assess viewing quality ofadvertisements exist, many include “pure” Javascript systems that sufferfrom natural restrictions of Javascript. As an example, when anadvertisement (or other media object) is placed within a frame (e.g.,iframe) embedded on a webpage, Javascript code (or other client-sidecode) served with the advertisement may be unable to determine certaininformation generally used assess viewing quality of the advertisement(e.g., whether or the extent to which the advertisement is viewable).For example, the Javascript code may not be able to determine thedimensions of the web page (on which the frame is embedded), whichportion of the webpage is within the browser viewport rendering thewebpage, or other information generally used to assess viewing qualityof the advertisement. As such, it is important not to rely solely onJavascript (or other client-code) served with advertisements to assessviewing quality.

FIG. 1 is a high level depiction of an exemplary system 100 in which aviewing quality assessment system 140 is deployed to assess viewingquality of media objects (e.g., advertisements), according to anembodiment of the present teaching. The exemplary system 100 includesusers 110, a network 120, an advertising delivery system 130, a viewingquality assessment system 140, content sources 160, or other components.The network 120 in system 100 can be a single network or a combinationof different networks. For example, a network can be a local areanetwork (LAN), a wide area network (WAN), a public network, a privatenetwork, a proprietary network, a Public Telephone Switched Network(PSTN), the Internet, a wireless network, a virtual network, or anycombination thereof. A network may also include various network accesspoints, e.g., wired or wireless access points such as base stations orInternet exchange points 120 a-120 n, through which a data source mayconnect to the network in order to transmit information via the network.

Users 110 may be of different types such as users connected to thenetwork via desktop connections (110 d), users connecting to the networkvia wireless connections such as through a laptop (110 c), a handhelddevice (110 a), or a built-in device in a motor vehicle (110 b). Theadvertising delivery system 130 may provide advertisements (or othercontent) for presentation to the users (e.g., on their user devices) viathe network 120. The advertisements may be provided in the form of mediaobjects (e.g., videos, animations, images, etc.) for presentation to theusers. In some embodiments, the viewing quality assessment system 140utilizes information received from the user devices to assess theviewing quality of media objects.

In an embodiment, a user device (or an application thereof) monitors oneor more delivery conditions of a media object that is served at the userdevice. These monitored delivery conditions include a condition of anetwork over which the media object was delivered (e.g., bandwidth ofthe network, latency of the network, etc.), a response time of the mediaobject (e.g., the amount of time between a request for the media objectand presentation of the media object), a frame rate of the media object(e.g., a frame rate at which the media object is rendered), acompression ratio of the media object, or other delivery condition.Based on the monitoring, the user device transmits information regardingthe delivery conditions of the media object to the viewing qualityassessment system 140, which then processes the delivery conditioninformation to estimate the viewing quality of the media object.

In an embodiment, upon receipt of information regarding one or moredelivery conditions of a media object (e.g., from a user device at whichthe media object is served), the viewing quality assessment system 140processes the delivery condition information to determine whether one ormore condition thresholds have been satisfied, and estimates the viewingquality of the media object based on whether the condition thresholdshave been satisfied. As an example, a delivery condition of the mediaobject that satisfies a condition threshold may indicate whether (or theextent to which) the media object is viewable to a user (e.g., that themedia object is “in-view,” that the media object is “out-of-view,”etc.). As another example, a delivery condition of the media object thatdoes not satisfy a condition threshold may indicate whether (or theextent to which) the media object is viewable to a user (e.g., that themedia object is “in-view,” that the media object is “out-of-view,”etc.).

In one use case, a media object (e.g., an advertisement) is delivered toa user device, which is then rendered on a browser (or otherapplication) at the user device. The browser is programmed to throttle aframe rate of the media object when the media object (or a certainpercentage thereof) is not within the browser viewport. As an example,although certain media objects (e.g., videos) are typically rendered onthe browser at frame rates of 24, 25, 30, 48, 50, or 60 frames persecond (fps) (e.g., when the media object is within the browserviewport), the browser throttles a media object to a frame rate of 1 fpswhen it detects that the media object is not within the browserviewport. The browser monitors and provides information regarding one ormore delivery conditions of the media object, such as the current framerate of the media object, to the viewing quality assessment system 140.The viewing quality assessment system 140, for example, then processesthe frame rate of the media object to determine whether the frame rateis greater than a particular frame rate threshold (e.g., 1 fps or otherframe rate less than common video frame rates).

If the frame rate is less than or equal to the frame rate threshold(e.g., which indicates that the browser throttled the media object), theviewing quality assessment system 140 determines that the media object(or a portion thereof) is currently not in within the browser viewport.Otherwise, the viewing quality assessment system 140 determines that themedia object (or a portion thereof) is currently within the browserviewport. The determination of whether the media object (or a portionthereof) is within the browser viewport may be utilized by viewingassessment system 140 to determine whether (or the extent to which) themedia object is viewable to a user. Other factors may, of course, beutilized to determine whether (or the extent to which) the media objectis within the browser viewport, as well as whether (or the extent towhich) the media object is viewable to a user.

In an embodiment, the viewing quality assessment system 140 need notnecessarily receive information identifying the actual frame rate (orabsolute values of other delivery conditions) of a media object that isserved at a user device to estimate the viewing quality of the mediaobject. As an example, in some embodiments, the viewing qualityassessment system 140 utilizes information regarding one or more statesof the media object to estimate the viewing quality of the media object.Upon receipt of the state information, for example, the viewing qualityassessment system 140 processes the state information to determinewhether the media object is in a state indicating that the media objectis viewable to a user, and estimates the viewing quality of the mediaobject based on whether the media object is in a state indicating thatthe media object is viewable to a user. The estimated viewing qualityfor the media object may, for instance, include “viewable to a user,”“not viewable to a user,” “an X percentage is viewable to a user,” orother estimation.

In one scenario, a media object (e.g., an advertisement) is delivered toa user device, which is then rendered on a browser (or otherapplication) at the user device. The browser is programmed to invoke athrottle-down event with respect to the media object when the mediaobject (or a certain percentage thereof) is not within the browserviewport. When a throttle-down event is invoked for the media object, aframe rate of the media object (or other parameter thereof) is throttleddown. As an example, although the media object is typically rendered onthe browser at a frame rate of 24 fps (or other frame rate) (e.g., whenthe media object is within the browser viewport), the browser throttlesdown the frame rate of the media object to a frame rate of 1 fps when itdetects that the media object is not within the browser viewport. Uponinvoking the throttle-down event, the browser provides informationindicating the throttle-down event (e.g., state information specifyingthat the throttle-down event was invoked for the media object) to theviewing quality assessment system 140. When the viewing qualityassessment system 140 processes the indication of the throttle-downevent, it determines that the media object (or a certain percentagethereof) is not currently within the browser viewport (and/or notviewable to a user). As such, by using such throttle event indications,the actual frame rate (or absolute values of other delivery conditions)of the media object need not necessarily be known by the viewing qualityassessment system 140 for it to estimate the viewing quality of themedia object.

In an embodiment, a user device (or an application thereof) monitors oneor more delivery conditions of objects associated with a media objectthat is served at the user device. In some embodiments, the associatedobjects are rendered at the user device with the media object (e.g.,behind the media object, in front of the media object, adjacent to themedia object, proximate to the media object, etc.). Based on themonitoring, the user device transmits information regarding the deliveryconditions of the associated objects to the viewing quality assessmentsystem 140, which then processes the delivery condition information toestimate the viewing quality of the associated objects and/or the mediaobject.

In an embodiment, upon receipt of information regarding deliveryconditions of objects associated with a media object (e.g., from a userdevice at which the media object is served), the viewing qualityassessment system 140 processes the delivery condition information todetermine whether one or more delivery conditions of the associatedobjects satisfy one or more condition thresholds. The viewing qualityassessment system 140 estimates, for each of the associated objects, theviewing quality of the associated object based on whether (or how manyof) the delivery conditions of the associated object satisfies thecondition thresholds. The estimated viewing quality for each associatedobject may, for instance, include “viewable to a user,” “not viewable toa user,” “an X percentage is viewable to a user,” or other estimation.The viewing quality assessment system 140 then estimates the viewingquality of the media object based on the estimated viewing quality ofthe associated objects. As an example, if the number of the associatedobjects estimated to be viewable to a user is greater than the number ofthe associated objects estimated to not be viewable to a user, thisindicates to the viewing quality assessment system 140 that the mediaobject (or at least a sufficient portion thereof) is viewable to theuser.

In an embodiment, a user device (or an application thereof) monitors oneor more states of objects associated with a media object that is servedat the user device. Based on the monitoring, the user device transmitsinformation regarding the states of the associated objects to theviewing quality assessment system 140, which then processes the stateinformation to estimate the viewing quality of the associated objectsand/or the media object. In an embodiment, upon receipt of the stateinformation, the viewing quality assessment system 140 processes thestate information to determine, for each of the associated objects,whether the associated object is in a state indicating that theassociated object is viewable to a user. The viewing quality assessmentsystem 140 estimates, for each of the associated objects, the viewingquality of the associated object based on whether the associated objectis in a state indicating that the associated object is viewable to auser. The estimated viewing quality for each associated object may, forinstance, include “viewable to a user,” “not viewable to a user,” “an Xpercentage is viewable to a user,” or other estimation. The viewingquality assessment system 140 then estimates the viewing quality of themedia object based on the estimated viewing quality of the associatedobjects. As an example, if the number of the associated objectsestimated to be viewable to a user is greater than the number of theassociated objects estimated to not be viewable to a user, thisindicates to the viewing quality assessment system 140 that the mediaobject (or at least a sufficient portion thereof) is viewable to theuser.

In one use case, a media object (e.g., an advertisement) is delivered toa user device, which is then rendered on a browser (or otherapplication) at the user device. One or more objects associated with themedia object is delivered to the user device and/or rendered on thebrowser at the user device. The browser is programmed to invoke, foreach of the associated objects, a throttle-down event with respect tothe associated object when the associated object (or a certainpercentage thereof) is not within the browser viewport. When athrottle-down event is invoked for an associated object, a frame rate ofthe associated object (or other parameter thereof) is throttled down. Asan example, although each associated object is typically rendered on thebrowser at a frame rate of 24 fps (or other frame rate) (e.g., when theassociated object is within the browser viewport), the browser throttlesdown the frame rate of the associated object to a frame rate of 1 fpswhen it detects that the associated object is not within the browserviewport. Upon invoking the throttle-down event, the browser providesinformation indicating the throttle-down event (e.g., state informationspecifying that the throttle-down event was invoked for the associatedobject) to the viewing quality assessment system 140.

When the viewing quality assessment system 140 processes the indicationof the throttle-down event, it determines that the associated object(for which the throttle-down event was invoked) is not currently withinthe browser viewport (and/or not viewable to a user). The viewingquality assessment system 140 estimates the viewing quality of the mediaobject based on the number of the associated objects not viewable to auser, the number of the associated objects viewable to a user, etc. Asan example, if the viewing quality assessment system 140 determines thata certain threshold number of the associated objects are not within thebrowser viewport, it determines that the media object (or at least acertain percentage thereof) is not within the browser viewport (and/ornot viewable to a user). As another example, the viewing qualityassessment system 140 estimates the extent to which the media object iswithin (or not within) the browser viewport (and/or viewable or notviewable to a user) based on the number of the associated objects notviewable to a user, the number of the associated objects viewable to auser, etc. In this way, by using such throttle event indications, theviewing quality assessment system 140 need not necessarily receiveinformation identifying the actual frame rates (or the absolute valuesof other delivery conditions) of the associated objects to estimate theviewing equality of the associated objects and/or the viewing quality ofthe media object.

The content sources 160 include multiple content sources 160 a, 160 b,160 n. A content source may correspond to a web page host correspondingto an entity, whether it be an individual, a business, or anorganization such as USPTO.gov; a content provider such as CNN.com andYahoo.com; or a content feed source such as Twitter or blogs. Both theadvertising delivery system 130 and the viewing quality assessmentsystem 140 may access information from any of the content sources 160 a,160 b, . . . 160 n. In an embodiment, for example, advertising deliverysystem 130 obtains content related to an advertisement from one or moreof content sources 160 a, 160 b, . . . 160 n to generate theadvertisement and/or provide for presentation with the advertisement toa user.

FIG. 2 is a high level depiction of an exemplary system 200 in which theviewing quality assessment system 140 is deployed to assess viewingquality of media objects (e.g., advertisements), according to anembodiment of the present teaching. The exemplary networked environment200 in this embodiment is similar to the exemplary networked environment100 in FIG. 1, except that the viewing quality assessment system 140 inthis embodiment connects to the network 120 via the advertising deliverysystem 130. For example, the viewing quality assessment system 140 mayserve as a back-end system of the advertising delivery system 130.

FIG. 3 depicts the inputs and output of the viewing quality assessmentsystem 140, according to an embodiment of the present teaching. Asshown, the viewing quality assessment system 140 receives, as input,information regarding one or more delivery conditions of a media object,objects associated with the media object, or other objects. The deliverycondition information includes: (1) one or more absolute values of abandwidth of a network over which an object is delivered, a latency ofthe network, a response time of the object, a frame rate of the object,a compression ratio of the object, etc.; (2) one or more relative valuesof a bandwidth of a network over which an object is delivered, a latencyof the network, a response time of the object, a frame rate of theobject, a compression ratio of the object, etc.; (3) adelivery-condition-related state of the object (e.g., throttled-downstate, throttled-up state, non-throttled state, etc.); or (4) otherinformation. By analyzing the delivery condition information, theviewing quality assessment system 140 can estimate viewing quality ofmedia objects (e.g., advertisements), and provide the viewing qualityfor the media objects for reporting to one or more business entities(e.g., to generate billing information and/or reports related to theviewing quality of the media objects).

FIG. 4 is a high level exemplary diagram of different deliveryconditions, according to an embodiment of the present teaching. Asillustrated in FIG. 4, delivery conditions of an object may includeconditions of a network (e.g., bandwidth, latency, etc.) over which theobject is delivered, a response time of the object, a frame rate of theobject, a compression ratio of the object, or other delivery condition.

FIG. 5 depicts a high level exemplary system diagram of the advertisingdelivery system 130 and the viewing quality assessment system 140,according to an embodiment of the present teaching. As shown, users 110submit requests for content (e.g., to load a webpage that includes anadvertisement), which may cause requests for advertisements to beprovided to the advertising delivery system 130. In response,advertising delivery system 130 provides one or more advertisements (orother objects) for presentation to users 110. User devices (or anapplication thereof) of users 110 render the advertisements and providedelivery condition information (or other information) to the viewingquality assessment system 140.

As depicted in FIG. 5, the viewing quality assessment system 140comprises a delivery condition evaluation unit 510, a viewing qualityevaluation unit 520, and a billing and/or reporting evaluation unit 530.In an embodiment, when information regarding one or more deliveryconditions of a media object (served at a user device) is received atthe viewing quality assessment system 140, delivery condition evaluationunit 510 processes the delivery condition information to determine thedelivery conditions of the media object, and provides its determinationsto viewing quality evaluation unit 520. The determinations (provided toviewing quality evaluation unit 520) may include a determination ofwhether the delivery conditions of the media object satisfy one or morecondition thresholds, a determination of a state of the media object, adetermination that a throttle event (e.g., a throttle-down event, athrottle-up event, etc.) was invoked with respect to the media object,or other determinations.

In an embodiment, when information regarding one or more deliveryconditions of one or more objects associated with the media object isreceived at the viewing quality assessment system 140, deliverycondition evaluation unit 510 processes the delivery conditioninformation to determine the delivery conditions of the associatedobjects, and provides its determinations to viewing quality evaluationunit 520. The determinations (provided to viewing quality evaluationunit 520) may include a determination of whether the delivery conditionsof an associated object satisfy one or more condition thresholds, adetermination of a state of the associated object, a determination thata throttle event (e.g., a throttle-down event, a throttle-up event,etc.) was invoked with respect to the associated object, or otherdeterminations.

In some embodiments, advertising delivery system 130 providesinformation regarding one or more advertising pricing parameters to theviewing quality assessment system 140. The advertising pricingparameters include, for example, one or more condition thresholds (e.g.,predetermined by advertising delivery system 130 in accordance with anagreement between an advertiser and its business customers) that governwhether (or how much) is to be charged for serving the media object. Inone use case, for example, if the compression ratio is higher than acertain compression rate threshold, the viewing quality of the mediaobject will likely be degraded below an acceptable quality of thecustomer. As such, the customer of the advertiser may be charged adiscounted rate (or not charged at all) for serving the media object ifthe compression ratio is higher than the compression ratio threshold. Inanother use case, if the latency of the network over which the mediaobject is delivered is higher than a certain latency threshold, and themedia object is a video advertisement, the playback of the videoadvertisement will likely load slowly and pause frequently. Thus, thecustomer of the advertiser may be charged a discounted rate (or notcharged at all) for serving the media object if the network latency ishigher than the latency threshold. In yet another use case, if the framerate of the media object is below a certain frame rate threshold at agiven time, the media object is likely either not within the viewport ofa browser at which it is rendered, or the rendering of the media objectis degraded below an acceptable quality of the customer. Therefore, thecustomer of the advertiser may be charged a discounted rate (or notcharged at all) for serving the media object if the frame rate of themedia object is below the frame rate threshold (e.g., depending on howlong or how often the frame rate of the media object is below the framerate threshold). Delivery condition evaluation unit 510 may utilize thecondition thresholds (provided by the advertising delivery system 130)or other condition thresholds to evaluate the delivery conditions of themedia object, and provide its evaluations to viewing quality evaluationunit 520 and/or billing and/or reporting evaluation unit 530.

Viewing quality evaluation unit 520 estimates a viewing quality of themedia object and provides its estimations to billing and/or reportingevaluation unit 530. As shown in FIG. 5, in some embodiments, viewingquality evaluation unit 520 provides its estimations to advertisingdelivery system 130, which may, for example, allow advertising system130 to perform its own analysis or generate its own reports regardingviewing quality of advertisements that it provides for presentation tousers 110.

In an embodiment, viewing quality evaluation unit 520 estimates theviewing quality of the media object based on the determinations(provided by delivery condition evaluation unit 510) related to thedelivery conditions of the media object. As an example, if a frame rateof the media object does not satisfy a predetermined frame ratethreshold (related to whether the media object rendered on a browser iswithin the browser viewport) at a given time, it is determined that themedia object was not within the browser viewport at the given time and,thus, not viewable to a user at the given time. As another example, ifframe rates of at least some of the objects associated with the mediaobject do not satisfy the predetermined frame rate threshold at a giventime, it is determined that those associated objects were not within thebrowser viewport at the given time and, thus, not viewable to a user atthe given time. Viewing quality evaluation unit 520 then estimates theviewing quality of the media object (e.g., whether or the extent towhich the media object is viewable to a user) based on the number of theassociated objects having frame rates that do not satisfy the frame ratethreshold, the number of the associated objects having frame rates thatsatisfy the frame rate threshold, etc.

In yet another example, the actual frame rate(s) (or absolute values ofother delivery conditions) need not necessarily be received (orotherwise known) by the viewing quality assessment system 140 for it toestimate the viewing quality of the media object. In one scenario, forexample, a browser (or other application) is programmed to invoke athrottle-down event when the media object falls outside of the browserviewport and to invoke a throttle-up event when the media object fallsback within the browser viewport. The throttle-down event causes themedia object to be rendered at a frame rate equal to or less than thepredetermined frame rate threshold (which the frame rate is compared todetermine whether the media object is within the browser viewport). Thethrottle-up event causes the media object to be rendered at a frame rategreater than the predetermined frame threshold. When the browser invokesa throttle event, an indication of the throttle event and/or otherinformation (e.g., a time at which the throttle event was invoked) isprovided to the viewing quality assessment system 140 in real-time(e.g., by the browser or other application) and/or stored for latertransmission to the viewing quality assessment system 140. Upon receiptof these throttle event indications, viewing quality evaluation unit 520processes the throttle event indications to determine whether the mediaobject was within the browser viewport at one or more given times. Theviewing quality evaluation unit 520 then estimates the viewing qualityof the media object based on whether the media object was within thebrowser viewport at the given times.

In another scenario, the browser is programmed to invoke a throttle-downevent with respect to an associated object of the media object when theassociated object falls outside of the browser viewport, and to invoke athrottle-up event when the associated object falls back within thebrowser viewport. The throttle-down event causes the associated objectto be rendered at a frame rate equal to or less than the predeterminedframe rate threshold (which the frame rate is compared to determinewhether the associated object is within the browser viewport). Thethrottle-up event causes the associated object to be rendered at a framerate greater than the predetermined frame threshold. When the browserinvokes a throttle event, an indication of the throttle event and/orother information (e.g., a time at which the throttle event was invoked)is provided to the viewing quality assessment system 140 in real-time(e.g., by the browser or other application) and/or stored for latertransmission to the viewing quality assessment system 140. Upon receiptof these throttle event indications, the viewing quality evaluation unit520 processes the throttle event indications to determine whether theassociated object was within the browser viewport. Viewing qualityevaluation unit 520 then estimates the viewing quality of the mediaobject based on the number of the associated objects within the viewportat one or more given times, the number of the associated objects outsidethe viewport at the given times, etc.

As depicted in FIG. 5, the advertising delivery system 130 providesadvertising pricing information, object information (e.g., objectdimensions, web page position, delivery conditions, etc.) to the viewingquality assessment system 140. In an embodiment, billing and/orreporting evaluation unit 530 processes such information (provided byadvertising delivery system 130), estimations provided by viewingquality evaluation unit 520, determinations provided by deliverycondition evaluation unit 510 (e.g., whether certain conditionthresholds were satisfied), or other information to: (1) determinebilling information; (2) generate reports related to the viewing qualityof the media object; or (3) perform other operations. In someembodiments, billing and/or reporting evaluation unit 530 provides thebilling information, the reports, or other information to ad metricsdatabase 540. Advertising delivery system 130 may, for example, obtaininformation from ad metrics database 540 for use in: (1) analyzing theeffectiveness of the media objects (e.g., advertisements) and/or itsplacements within various websites; (2) selecting new media objects forpresentation to users 110; or (3) performing other operations.

FIG. 6 is a flowchart of an exemplary process of assessing billinginformation related to advertisements, according to an embodiment of thepresent teaching. As shown, at 610, information regarding a deliverycondition of an advertisement (that is served at a user device) isobtained. The delivery condition information includes: (1) one or moreabsolute values of a bandwidth of a network over which an object isdelivered, a latency of the network, a response time of the object, aframe rate of the object, a compression ratio of the object, etc.; (2)one or more relative values of a bandwidth of a network over which anobject is delivered, a latency of the network, a response time of theobject, a frame rate of the object, a compression ratio of the object,etc.; (3) a delivery-condition-related state of the object (e.g.,throttled-down state, throttled-up state, non-throttled state, etc.); or(4) other information.

At 620, viewing quality of the advertisement is estimated based on thedelivery conditions of the media object. In an embodiment, the viewingquality of the advertisement is estimated based on whether the deliveryconditions of the media object satisfy one or more condition thresholds.In another embodiment, the viewing quality is estimated by determiningwhether (or the extent to which) the advertisement is viewable to a user(of the user device at which the media object is served) based on thedelivery conditions of the media object.

At 630, information regarding advertising pricing parameters related tothe advertisement is obtained. Advertising pricing parameters includeone or more rates at which to charge for serving the advertisement, oneor more condition thresholds that are to be satisfied (e.g., by deliveryconditions, viewing quality, etc., of the media object) beforerespective rates can be charged, or other parameters.

At 640, billing information related to the advertisement is determinedbased on the viewing quality of the advertisement and the advertisingpricing parameters. In some embodiments, a report (indicating theviewing quality of the advertisement) is generated based on the viewingquality of the advertisement. As an example, a report indicating viewingquality associated with at least a portion of an advertising campaignfor an advertising customer is generated based on the viewing quality ofthe advertisement and at least one other advertisement, where theadvertising campaign includes the advertisement and the otheradvertisement.

FIG. 7 is a high level exemplary system diagram of a delivery conditionevaluation unit, according to an embodiment of the present teaching. Asillustrated, the delivery condition evaluation unit 510 comprises anevaluation input manager 710, a network condition evaluation unit 720, aresponse time evaluation unit 730, a frame rate evaluation unit 740, acompression rate evaluation unit 750, and an evaluation output manager760. As an example, upon receipt of input (e.g., object identifiers,delivery conditions, condition thresholds, etc.), evaluation inputmanager 720 processes the input to determine the appropriate componentsto which one or more portions of the input are to be provided. In oneuse case, an object's identifier, information regarding a condition of anetwork over which the object is delivered, and/or information regardinga network condition threshold is provided to the network conditionevaluation unit 720. In another use case, an object's identifier,information regarding a response time of the object, and/or informationregarding a response time threshold is provided to the response timeevaluation unit 730. In yet another use case, an object's identifier,information regarding a frame rate of the object, and/or informationregarding a frame rate threshold is provided to the frame rateevaluation unit 740. In a further use case, an object's identifier,information regarding a compression rate of the object, and/orinformation regarding a compression rate threshold is provided to thecompression rate evaluation unit 750. Upon processing such information,respectively, the various evaluation units 720, 730, 740, and 750provide their evaluations (e.g., which delivery conditions of the objectsatisfy the respective condition thresholds, whether the object wasthrottled, etc.) to the evaluation output manager 760. The evaluationoutput manager 760 processes the evaluations to determine theappropriate components (e.g., outside the delivery condition evaluationunit 510) to which one or more of the evaluations are to be provided.

FIGS. 8A and 8B illustrate exemplary presentations of advertisements,according to different embodiments of the present teaching. As shown inFIG. 8A, an advertisement 810 may be provided for presentation byproviding a single media object (e.g., a video advertisement) forpresentation to a user at a given time. As discussed, in one use case,the viewing quality of the advertisement is estimated based on one ormore delivery conditions of the advertisement at one or more giventimes. For example, the viewing quality of the advertisement isestimated based on a delivery condition of a first portion of theadvertisement (e.g., a portion corresponding to a first time interval ofthe video advertisement) at a first time, a delivery condition of asecond portion of the advertisement (e.g., a portion corresponding to asecond time interval of the video advertisement), and so on.

As shown in FIG. 8B, an advertisement 820 may be provided forpresentation by providing respective portions of multiple media objects830 at a given time. The multiple media objects (e.g., multiple videos)may be provided for presentation such that they appear to be a singlevideo object (e.g., a single video) from the perspective of a user orsuch that they appear to be part of a single advertisement. In someembodiments, a browser (or other application) of a user device (at whichan advertisement is served) is programmed to invoke a throttle-downevent with respect to an object only when the object is completelyoutside the browser viewport, and to invoke a throttle-up event withrespect to the object when the object is at least partially within thebrowser viewport. In such embodiments, for example, an advertisementthat is represented by a single media object at a given time (e.g.,advertisement 810 of FIG. 8A) would only be throttled down when theadvertisement (or the single media object) is completely outside thebrowser viewport. As such, an indication that the advertisement is notthrottled at a given time may not by itself be a good indication of howmuch of the advertisement is viewable to a user at the given time.However, an advertisement that is represented by multiple media objectsat a given time (e.g., advertisement 820 of FIG. 8B) can have someportions of the advertisement throttled down and some portions of theadvertisement not throttled down at the same time, enabling indicationsof which particular portions of the advertisement are outside thebrowser viewport. Thus, such indications together may be a betterindication of how much of the advertisement is viewable to a user at agiven time (e.g., as compared to advertisement 810 of FIG. 8A).

FIG. 9 is a flowchart of an exemplary process of assessing viewingquality of an advertisement based on delivery conditions of portions ofthe advertisement, according to an embodiment of the present teaching.As shown, at 910, information regarding a delivery condition of aportion of an advertisement is obtained. The advertisement may, forexample, be an advertisement as described with respect to FIG. 8A (e.g.,single media object at a given time) or an advertisement as describedwith respect to FIG. 8B (e.g., multiple media objects at a given time).

At 920, a determination of whether there is information regardingdelivery conditions of other portions of the advertisement that has notbeen (and should be) obtained is made. The other portions may, forexample, include: (1) portions of the advertisement that are rendered ata different time than one or more portions of the advertisement forwhich delivery condition information has been obtained (e.g., differenttime interval of a video advertisement); or (2) portions of theadvertisement that are rendered the same time as the advertisementportions for which delivery condition information has been obtained.

If such another portion of the advertisement exists, informationregarding a delivery condition of a next portion of the advertisement isobtained at 910. Otherwise, at 930, viewing quality of the advertisementis estimated based on the delivery conditions of the portions of theadvertisement.

FIG. 10 is a flowchart of an exemplary process of assessing viewingquality of an advertisement based on a frame rate of the advertisement,according to an embodiment of the present teaching. As shown, at 1010,information regarding a frame rate of an advertisement is obtained.

At 1020, viewing quality of the advertisement is estimated based on theframe rate of the advertisement. As an example, the viewing quality ofthe advertisement is estimated by determining whether (or the extent towhich) the advertisement is viewable to a user based on the frame rateof the advertisement.

At 1030, a report indicating the viewing quality of the advertisement isgenerated. As an example, a report indicating viewing quality associatedwith at least a portion of an advertising campaign for an advertisingcustomer is generated based on the viewing quality of the advertisementand at least one other advertisement, where the advertising campaignincludes the advertisement and the other advertisement.

FIG. 11 is a flowchart of an exemplary process of assessing viewingquality of a media object based on a state of an object associated withthe media object, according to an embodiment of the present teaching. Asshown, at 1110, information regarding a state of an object associatedwith a media object (that is served at a user device) is obtained. As anexample, the state of the associated object may be a throttled-downstate, a throttled-up state, a non-throttled state, or other state. Theassociated object may be a supplemental object for tracking and/ormonitoring aspects of the media object.

At 1120, viewing quality of the associated object is estimated based onthe state of the associated object, without receiving a frame rate ofthe object. At 1130, viewing quality of the media object is estimatedbased on the viewing quality of the associated object. As an example, ifthe number of associated objects estimated to be viewable to a user isgreater than the number of associated objects estimated to not beviewable to a user, then this is an indication that the media object (orat least a sufficient portion thereof) is viewable to the user.

At 1140, billing information related to delivery of the media object isdetermined based on the viewing quality of the associated object. As anexample, an advertising customer is charged for the serving of the mediaobject based on whether (or the extent to which) the media object isdeemed to be viewable to a user. The determination of whether (or theextent to which) the media object is deemed to be viewable is based onthe viewing quality of the associated object and/or other associatedobjects. As another example, an advertising customer is charged for theserving of the media object relative to the number of the associatedobjects that are estimated to be viewable, relative to the number ofassociated objects that are estimated not to be viewable, etc.

FIG. 12 is a high level exemplary system diagram of a viewing qualityevaluation unit, according to an embodiment of the present teaching. Asshown, the viewing quality evaluation unit 520 comprises an objectpresentation evaluation unit 1210 and a viewing quality validation unit1220. In an embodiment, object presentation evaluation unit 1210processes object identifiers, dimension information (e.g., object heightand width, dimension of a webpage that includes the object, etc.),position information (e.g., object position within the webpage, whichportion of the webpage is within the browser viewport, etc.), or otherinformation (e.g., received from the user device) to optimize theinformation for the viewing quality validation unit 1220 (which utilizesthe information to determine whether (or the extent to which) a renderedobject is viewable).

In some embodiments, Javascript (or other programming language) may beutilized to obtain and/or process the object identifier, the dimensioninformation, the position information, or other information to determinewhether (or the extent to which) a rendered object is viewable to auser. However, due to the natural limitations of Javascript, validationof the viewing quality of a rendered object purely using informationobtained from Javascript served with the object may not be accurate. Asan example, such validation purely using Javascript may fail to indicatean object rendered within the viewport of a browser as viewable to auser when the browser is not the active window that the user iscurrently using even, if the object is clearly in-view from theperspective of the user. As such, in an embodiment, feedback from thebrowser that does not necessarily rely on Javascript (e.g., informationregarding a delivery condition of the object) may be utilized tosupplement or replace information obtained via Javascript operations.

In an embodiment, the viewing quality validation unit 1220 processescondition threshold satisfaction indications, throttle eventindications, or other information to determine whether (or the extent towhich) a rendered object is viewable to a user.

FIG. 13 illustrates an exemplary presentation of an advertisement andits associated objects, according to different embodiments of thepresent teaching. As shown, an advertisement 1310 is provided forpresentation to a user and one or more associated objects 1320 areprovided behind the advertisement 1310 (as indicated by dotted lines) atspecific positions relative to the advertisement 1310. In one use case,a browser (or other application) that renders the advertisement 1310 andthe associated objects 1320 is programmed to invoke a throttle-downevent with respect to an object (e.g., the advertisement 1310, theassociated objects, etc.) when the object is completely outside thebrowser viewport, and to invoke a throttle-up event with respect to anobject when the object is at least partially within the browserviewport. As such, when a portion of the advertisement 1310 fallsoutside the browser viewport (without the entirety of the advertisementfalling outside the browser viewport), the browser may not invoke athrottle-down event for the advertisement 1310. However, when a portionof the advertisement falls outside the browser viewport, the browserinvokes throttle-down events for any associated objects 1320 that arecompletely within the portion of the advertisement outside the browserviewport. Viewing quality assessment system 140 obtains indications ofthe throttle-down events from the browser (or other application), andprocesses the throttle-down event indications to determine whether (orthe extent to which) the advertisement 1310 is viewable to a user.

In another use case, with respect to FIG. 13, if viewing qualityassessment system 140 determines (e.g., based on the throttle events)that at most 2 of the 5 associated objects 1320 are throttled down(e.g., slowed frame rate) at a given time, it may mark the advertisement1310 as viewable to a user at the given time. Otherwise, if viewingquality assessment system 140 determines that at least 3 of the 5associated objects are throttled down at the given time, it may mark theadvertisement 1310 as not viewable to a user at the given time. Otherfactors may, of course, be utilized to estimate whether (or the extentto which) the advertisement is viewable to a user.

In a further use case, with respect to FIG. 13, the viewing qualityassessment system may repeat these estimations at multiple given timesto estimate the time period (e.g., absolute time period, relative timeperiod, etc.) during which the advertisement 1310 is deemed to beviewable to a user (or during which the advertisement is deemed not tobe viewable to a user). The time period may then be utilized forreporting (e.g., information related to the viewing quality of theadvertisement) to one or more business entities. For example, the timeperiod is utilized to determine billing information related to deliveryof the advertisement 1310 and/or generate a report indicating theviewability of the advertisement 1310. It should be appreciated that, inother embodiments, any number, dimension, or location of associatedobjects 1320 may be used to assist in monitoring the viewing quality(e.g., viewability) of the advertisement 1310 (e.g., greater number ofassociated objects 1320 to increase the accuracy of the estimations ofthe viewing quality of the advertisement 1310).

FIG. 14 is a flowchart of an exemplary process of assessing viewingquality of a media object based on a related throttle event, accordingto an embodiment of the present teaching. As shown, at 1410, informationindicating at least one throttle event with respect to one or moreobjects associated with a media object (that is served at a user device)is obtained. The throttle event may, for example, include athrottle-down event, a throttle-up event, or other event.

At 1420, viewing quality of the associated objects is estimated based onthe throttle event. As an example, if an indication of a throttle-downevent with respect to an associated object is obtained, the associatedobject may be determined to be outside the browser viewport. On theother hand, if an indication of a throttle-up event with respect to anassociated object is obtained, the associated object may be determinedto be within the browser viewport.

At 1430, viewing quality of the media object is estimated based on theviewing quality of the associated objects. As an example, theindications of the throttle-down events and/or the throttle-up eventsare used to estimate whether (or the extent to which) the media object(with which the objects are associated) is within the browser viewport(and/or is viewable to a user).

At 1440, billing information related to delivery of the media object isdetermined based on the viewing quality of the associated objects. As anexample, an advertising customer is charged for the serving of the mediaobject based on whether (or the extent to which) the media object isdeemed to be viewable to a user. The determination of whether (or theextent to which) the media object is deemed to be viewable is based onthe viewing quality of the associated object and/or other associatedobjects. As another example, an advertising customer is charged for theserving of the media object relative to the number of the associatedobjects that are estimated to be viewable, relative to the number ofassociated objects that are estimated not to be viewable, etc.

FIG. 15 is a flowchart of an exemplary process of assessing viewingquality of advertisements, according to an embodiment of the presentteaching. As shown, in some embodiments, Javascript is still used toassess viewing quality of media objects (e.g., advertisements). At 1510,an advertisement is rendered on a browser (or other application) on auser device. At 1520, Javascript access is tested, and, at 1530, adetermination of whether Javascript can be executed is effectuated. IfJavascript cannot be executed, then, at 1540, the impression of theadvertisement is flagged as unmeasurable and/or not viewable. Otherwise,at 1550, viewability validation is performed (as described here), and,at 1560, a determination of whether the advertisement is viewable iseffectuated. If the advertisement is deemed not to be viewable, then, at1570, the impression of the advertisement is flagged as unmeasurableand/or not viewable. Otherwise, at 1580, the impression of theadvertisement is flagged as measurable and viewable.

FIG. 16 depicts an architecture of a mobile device that can be used torealize the present teaching as a specialized system.

In this example, the user device on which a media object (e.g., anadvertisement) is presented is a mobile device 1600, such as but notlimited to, a smart phone, a tablet, a music player, a handled gamingconsole, or a global positioning system (GPS) receiver. The mobiledevice 1600 in this example includes one or more central processingunits (CPUs) 1602, one or more graphic processing units (GPUs) 1604, adisplay 1606, a memory 1608, a communication platform 1610 (such as awireless communication module), storage 1612, and one or moreinput/output (I/O) devices 1614. Any other suitable component, such asbut not limited to a system bus or a controller (not shown), may also beincluded in the mobile device 1600. As shown in FIG. 16, a mobileoperating system 1616, e.g., iOS, Android, Windows Phone, etc., and oneor more applications 1618 may be loaded into the memory 1608 from thestorage 1612 in order to be executed by the CPU 1602. The applications1618 may include a browser or other application that enables a user toaccess content (e.g., advertisements or other content), providespresentations of content to users, monitors user activities related topresented content (e.g., whether a user has viewed an advertisement,whether the user interacted with the advertisement in other ways, etc.),reports events (e.g., throttle events), or performs other operations.

To implement the present teaching, computer hardware platforms may beused as the hardware platform(s) for one or more of the elementsdescribed herein. The hardware elements, operating systems, andprogramming languages of such computers are conventional in nature, andit is presumed that those skilled in the art are adequately familiartherewith to adapt those technologies to implement the processingessentially as described herein. A computer with user interface elementsmay be used to implement a personal computer (PC) or other type of workstation or terminal device, although a computer may also act as a serverif appropriately programmed. It is believed that those skilled in theart are familiar with the structure, programming, and general operationof such computer equipment and as a result the drawings should beself-explanatory.

FIG. 17 depicts an architecture of a computer based on which the presentteaching can be realized as a specialized system for assessing viewingquality of media objects. The computer may be a general-purpose computeror a special purpose computer. This computer 1700 can be used toimplement any components of the viewing quality assessment architectureas described herein. Different components of the system in the presentteaching can all be implemented on one or more computers such ascomputer 1700, via its hardware, software program, firmware, or acombination thereof. Although only one such computer is shown, forconvenience, the computer functions relating to the target metricidentification may be implemented in a distributed fashion on a numberof similar platforms, to distribute the processing load.

The computer 1700, for example, includes COM ports 1702 connected to andfrom a network connected thereto to facilitate data communications. Thecomputer 1700 also includes a central processing unit (CPU) 1704, in theform of one or more processors, for executing program instructions. Theexemplary computer platform includes an internal communication bus 1706,program storage and data storage of different forms, e.g., disk 1708,read only memory (ROM) 1710, or random access memory (RAM) 1712, forvarious data files to be processed and/or communicated by the computer,as well as possibly program instructions to be executed by the CPU. Thecomputer 1700 also includes an I/O component 1714, supportinginput/output flows between the computer and other components thereinsuch as user interface elements 1716. The computer 1700 may also receiveprogramming and data via network communications.

Hence, aspects of the method of assessing viewing quality of mediaobjects (e.g., advertisements), as outlined above, may be embodied inprogramming. Program aspects of the technology may be thought of as“products” or “articles of manufacture” typically in the form ofexecutable code and/or associated data that is carried on or embodied ina type of machine readable medium. Tangible non-transitory “storage”type media include any or all of the memory or other storage for thecomputers, processors or the like, or associated modules thereof, suchas various semiconductor memories, tape drives, disk drives and thelike, which may provide storage at any time for the softwareprogramming.

All or portions of the software may at times be communicated through anetwork such as the Internet or various other telecommunicationnetworks. Such communications, for example, may enable loading of thesoftware from one computer or processor into another. Thus, another typeof media that may bear the software elements includes optical,electrical, and electromagnetic waves, such as used across physicalinterfaces between local devices, through wired and optical landlinenetworks and over various air-links. The physical elements that carrysuch waves, such as wired or wireless links, optical links or the like,also may be considered as media bearing the software. As used herein,unless restricted to tangible “storage” media, terms such as computer ormachine “readable medium” refer to any medium that participates inproviding instructions to a processor for execution.

Hence, a machine readable medium may take many forms, including but notlimited to, a tangible storage medium, a carrier wave medium or physicaltransmission medium. Non-volatile storage media include, for example,optical or magnetic disks, such as any of the storage devices in anycomputer(s) or the like, which may be used to implement the system orany of its components as shown in the drawings. Volatile storage mediainclude dynamic memory, such as a main memory of such a computerplatform. Tangible transmission media include coaxial cables; copperwire and fiber optics, including the wires that form a bus within acomputer system. Carrier-wave transmission media can take the form ofelectric or electromagnetic signals, or acoustic or light waves such asthose generated during radio frequency (RF) and infrared (IR) datacommunications. Common forms of computer-readable media thereforeinclude for example: a floppy disk, a flexible disk, hard disk, magnetictape, any other magnetic medium, a CD-ROM, DVD or DVD-ROM, any otheroptical medium, punch cards paper tape, any other physical storagemedium with patterns of holes, a RAM, a PROM and EPROM, a FLASH-EPROM,any other memory chip or cartridge, a carrier wave transporting data orinstructions, cables or links transporting such a carrier wave, or anyother medium from which a computer can read programming code and/ordata. Many of these forms of computer readable media may be involved incarrying one or more sequences of one or more instructions to aprocessor for execution.

Those skilled in the art will recognize that the present teachings areamenable to a variety of modifications and/or enhancements. For example,although the implementation of various components described above may beembodied in a hardware device, it can also be implemented as a softwareonly solution. In addition, the components of the system as disclosedherein can be implemented as a firmware, firmware/software combination,firmware/hardware combination, or a hardware/firmware/softwarecombination.

While the foregoing has described what are considered to be the bestmode and/or other examples, it is understood that various modificationsmay be made therein and that the subject matter disclosed herein may beimplemented in various forms and examples, and that the teachings may beapplied in numerous applications, only some of which have been describedherein. It is intended by the following claims to claim any and allapplications, modifications and variations that fall within the truescope of the present teachings.

We claim:
 1. A method for assessing viewing quality of a media object,the method being implemented on a machine having at least one processor,storage, and a communication platform, the method comprising: receivinginformation indicating a throttle event with respect to each of one ormore supplemental objects associated with a media object, the mediaobject being served at a user device, the one or more supplementalobjects being overlaid on or underlaid behind the media object on a webpage loaded at the user device, wherein a throttle event throttles aparameter in connection with a supplemental object associated with themedia object; estimating viewing quality of the one or more supplementalobjects, that are overlaid on or underlaid behind the media object,based on the throttle event with respect to each of the one or moresupplemental objects; and estimating viewing quality of the media objectbased on the viewing quality of the one or more supplemental objectsthat are overlaid on or underlaid behind the media object.
 2. The methodof claim 1, wherein the parameter comprises at least one of a frame rateor a compression ratio.
 3. The method of claim 1, wherein the mediaobject comprises a video advertisement without the one or moresupplemental objects, and the one or more supplemental objects areoverlaid on or underlaid behind the video advertisement, and wherein thestep of estimating of estimating the viewing quality of the media objectcomprises estimating the viewing quality of the video advertisementbased on the viewing quality of the one or more supplemental objectsthat are overlaid on or underlaid behind the video advertisement.
 4. Themethod of claim 1, further comprising: determining billing informationrelated to delivery of the media object based on the viewing quality ofthe one or more supplemental objects.
 5. The method of claim 1, whereinthe media object comprises at least one of a video or animation.
 6. Themethod of claim 1, wherein the step of estimating the viewing quality ofthe one or more supplemental objects comprises processing the receivedthrottle event information to estimate, for each of the one or moresupplemental objects, the viewing quality of the supplemental objectbased on whether the supplemental object is viewable to a user.
 7. Themethod of claim 1, wherein a browser application at the user device isconfigured to invoke a throttle event for a supplemental objectassociated with the media object based one or more predefined criteria.8. The method of claim 1, further comprising: estimating viewing qualityof one or more other supplemental objects associated with the mediaobject, the one or more other supplemental objects being rendered withrespect to the media object on the web page, wherein the step ofestimating the viewing quality of the media object comprises estimatingthe viewing quality of the media object based on how many of the one ormore supplemental objects and the one or more other supplemental objectshave a throttle event invoked for the respective supplemental object ata given time.
 9. The method of claim 1, wherein a plurality ofsupplemental objects associated with the media object is rendered withrespect to the media object on the web page, the one or moresupplemental objects are at least a part of the plurality ofsupplemental objects, and at least one of the plurality of supplementalobjects is rendered at a location on the web page different from alocation on the web page at which another one of the plurality ofsupplemental objects is rendered.
 10. A system for assessing viewingquality of a media object, comprising: a delivery condition evaluationunit configured to receive information indicating a throttle event withrespect to each of one or more supplemental objects associated with amedia object, the media object being served at a user device, the one ormore supplemental objects being overlaid on or underlaid behind themedia object on a web page loaded at the user device, wherein a throttleevent throttles a parameter in connection with a supplemental objectassociated with the media object; and a viewing quality evaluation unitconfigured to: estimate viewing quality of the one or more supplementalobjects, that are overlaid on or underlaid behind the media object,based on the throttle event with respect to each of the one or moresupplemental objects; and estimate viewing quality of the media objectbased on the viewing quality of the one or more supplemental objectsthat are overlaid on or underlaid behind the media object.
 11. Thesystem of claim 10, wherein the parameter comprises at least one of aframe rate or a compression ratio.
 12. The system of claim 10, whereinthe media object comprises a video advertisement without the one or moresupplemental objects, and the one or more supplemental objects areoverlaid on or underlaid behind the video advertisement, and wherein thestep of estimating of estimating the viewing quality of the media objectcomprises estimating the viewing quality of the video advertisementbased on the viewing quality of the one or more supplemental objectsthat are overlaid on or underlaid behind the video advertisement. 13.The system of claim 10, further comprising: a billing and/or reportingevaluation unit configured to determine billing information related todelivery of the media object based on the viewing quality of the one ormore supplemental objects.
 14. The system of claim 10, wherein the mediaobject comprises at least one of a video or animation.
 15. The system ofclaim 10, wherein the viewing quality evaluation unit is configured toestimate the viewing quality of the one or more supplemental objects byprocessing the received throttle event information to estimate, for eachof the one or more supplemental objects, the viewing quality of thesupplemental object based on whether the supplemental object is viewableto a user.
 16. The system of claim 10, wherein a browser application atthe user device is configured to invoke a throttle event for asupplemental object associated with the media object based one or morepredefined criteria.
 17. The system of claim 10, the viewing qualityevaluation unit is configured to estimate viewing quality of one or moreother supplemental objects associated with the media object, the one ormore other supplemental objects being rendered with respect to the mediaobject on the web page, and wherein the viewing quality evaluation unitis configured to estimate the viewing quality of the media object byestimating the viewing quality of the media object based on how many ofthe one or more supplemental objects and the one or more othersupplemental objects have a throttle event invoked for the respectivesupplemental object at a given time.
 18. The system of claim 10, whereina plurality of supplemental objects associated with the media object isrendered with respect to the media object on the web page, the one ormore supplemental objects are at least a part of the plurality ofsupplemental objects, and at least one of the plurality of supplementalobjects is rendered at a location on the web page different from alocation on the web page at which another one of the plurality ofsupplemental objects is rendered.
 19. A method for assessing viewingquality of a media object, the method being implemented on a machinehaving at least one processor, storage, and a communication platform,the method comprising: receiving information indicating a throttle eventwith respect to each of one or more supplemental objects associated witha video, the video being an object separate from the one or moresupplemental objects and served at a user device, the one or moresupplemental objects being rendered with respect to the video on a webpage loaded at the user device, wherein a throttle event throttles aparameter in connection with a supplemental object associated with thevideo; estimating viewing quality of the one or more supplementalobjects based on the throttle event with respect to each of the one ormore supplemental objects; and estimating viewing quality of the videobased on the viewing quality of the one or more supplemental objects.20. The method of claim 19, wherein the one or more supplemental objectsare overlaid on or underlaid behind the video on the web page, whereinthe parameter comprises at least one of a frame rate or a compressionratio such that a throttle event throttles a frame rate or a compressionrate in connection with a supplemental object associated with the video,wherein the step of estimating the viewing quality of the one or moresupplemental objects comprises processing the received throttle eventinformation to estimate, for each of the one or more supplementalobjects, the viewing quality of the supplemental object based on whetherthe supplemental object is viewable to a user, wherein a browserapplication at the user device is configured to invoke a throttle eventfor a supplemental object associated with the video based one or morepredefined criteria, and wherein a plurality of supplemental objectsassociated with the video is rendered with respect to the video on theweb page, the one or more supplemental objects are at least a part ofthe plurality of supplemental objects, and at least one of the pluralityof supplemental objects is rendered at a location on the web pagedifferent from a location on the web page at which another one of theplurality of supplemental objects is rendered.